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Description 

TECHNICAL FIELD 

[0001 ] The present Invention relates to an image rec- 
ognition system, and in particular to an image recogni- 
tion system for detecting liquid drops such as rain water 
drops deposited on a transparent member such as a 
windshield. 

BACKGROUND OF THE INVENTION 

[0002] Water drop sensors are conventionally known 
for automatically operating a windshield wiper upon de- 
tecting rain drops (water drops) deposited on the sur- 
face of a windshield of a vehicle. They include those us- 
ing a base board which allows detection of water drop 
deposition by a change in the electric resistance of the 
base board, and those capturing and recognizing an im- 
age of the glass surface from inside the vehicle to detect 
the presence of water drops thereon. 
[0003] These relying on the detection of a change in 
the electric resistance for detection of water drops have 
the disadvantage of exposing a part of an electric circuit 
to the exterior so that the contamination of the exposed 
part and the need for regular maintenance create a 
problem. Therefore, water drop detection systems 
based on image recognition have been attracting a 
greater attention. 

[0004] However, when an attempt is made to capture 
the image of the surface of transparent glass, the image 
of the outside view is inevitably included in the surface 
image, and separating the water drop image from the 
overall image creates a problem. In order to recognize 
only the image of the water drops deposited on the sur- 
face of glass, it is necessary to increase the depth-of- 
field of the camera, but it in turn requires the distance 
between the camera and the glass to be reduced. How- 
ever, given the angle-of -field of the camera, as the cam- 
era is brought closer to the glass, the view range of the 
camera progressively diminishes. This prevents a high- 
sensitivity and high-precision detection of water drops. 
Glass surface often has dust and bird droppings depos- 
ited thereon, and may have scratches thereon. The 
presence of these surface contaminations may also pre- 
vent accurate detection of water drops because these 
contaminations may be easily confused with water 
drops. It has therefore been desired to achieve a high- 
sensitivity and high-precision detection of water drops 
with a wide view range image recognition. 
[0005] JP 63038058 that forms the closest prior art 
discloses an image sensor disposed on the inner sur- 
face of windshield for detecting a rain drop attached in 
the wiper range on the outer surface of the windshield. 
The sensor generates a series of images detection sig- 
nals in response to the received image. Values in each 
series of image detection signals produced by the image 
sensor for every predetermined time are averaged for 



each series by an averaging means and set as averaged 
data in sequence. An output signal necessary for actu- 
ating a windslhield wiper is generated from an output 
signal generating means so attached raindrops are 

5 wiped by action of the windshield wiper only when the 
averaged data are different from each other. 
[0006] JP051 236530 discloses an instrument for 
measuring and processing spraying area of liquid. Liq- 
uid is sprayed from a washer nozzle onto a transparent 

10 inspection board, the front surface of which can be seen 
from the rear side. Images of the liquid on the board are 
taken with a CCD camera from the rear side. An image 
processor produces a still picture based on the picture 
signal of the camera. 

15 [0007] GB 2208434A discloses a device for detecting 
water droplets on a vehicle window. The device com- 
prises a light source which emits light rays towards a 
window and on which is fitted a diffractive optical ele- 
ment. The diffractive optical element conveys the light 

20 rays through the window towards a photodetector which 
generates a signal corresponding to the intensity of light 
incident on it. 

[0008] DE19700665A published 24 July 1997 also 
discloses a device for detecting material on a wind- 
25 screen. The device comprises a light source which 
transmits light through a transparent wall to a detector. 
Image processing means process the image received 
by the detector to determine the density of raindrops on 
the screen. 

30 [0009] All of the devices disclosed by the above prior 
art cannot accurately extract an image of a liquid drop 
deposited on the surface of the transparent member 
from images of solid members and external objects. 

35 BRIEF SUMMARY OF THE INVENTION 

[0010] In view of such problems of the prior art, a pri- 
mary object of the present invention is to provide an im- 
age recognition system which can be used for detecting 

40 liquid drops deposited on a transparent member both 
accurate and sensitive manner. 
[0011] A second object of the present invention is to 
provide an image recognition system which can accu- 
rately distinguish liquid drops deposited on a transpar- 

45 ent member from solid depositions on the transparent 
member. 

[0012] A third object of the present invention is to pro- 
vide an image recognition system which can accurately 
extract an image of liquid drops deposited on a surface 
so of a transparent member from images of external ob- 
jects. 

[0013] A fourth object of the present invention is to 
provide an image recognition system which can be used 
for accurately detecting rain drops on a windshield. 
55 [001 4] According to the present invention there is pro- 
vided an image recognition system for detecting liquid 
drops deposited on a surface of a transparent member 
comprising: 



3 



EP0 832 798 B1 



4 



a camera for capturing an image; and 

means for determining the presence of liquid drops 

from the image captured by the camera, 

characterised in that 5 

the system further comprises a translucent screen 
placed over the transparent or a part thereof; and 
the camera is for capturing an image formed on said 
screen from a reverse surface of said transparent 10 
member. 

[0015] Owing to the use of the translucent screen, the 
only objects Immediately nearthe screen such as liquid 
drops deposited on the transparent member are cap- is 
tured by the camera while the objects remote from the 
transparent member are disregarded. Also, solid 
smears deposited on the transparent member can be 
distinguished from liquid drops by making use of the dif- 
ference in optical properties of these two different ob- 
jects. 

[001 6] The screen may consist of a translucent sheet 
member placed over the inner and outer surface of the 
transparent member. The sheet member may also be 
placed between two layers of the transparent member. 
The translucent sheet member may consist of a frosted 
sheet member, a milky sheet member or a semi-trans- 
parent member as long as it allows detection of objects 
immediately near the screen while dispersing the light 
from objects remote from the transparent member. It is 
also possible to achieve a similar result by directly frost- 
ing a surface of the transparent member, for instance by 
physical or chemical means. 
[0017] According to a preferred embodiment of the 
present invention, the transparent member consists of 
a windshield glass sheet separating an interior of a ve- 
hicle from outside so that rain water drops deposited on 
the glass sheet may be detected. The camera may be 
placed behind a rearview mirror, while the screen is 
placed on an inner surface of the windshield glass sheet 
opposite the camera. Alternatively, the screen may be 
placed on a lower edge of on an inner surface of the 
windshield glass sheet while the camera is mount on a 
part of a dashboard opposing the screen. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0018] Now the present invention is described in the 
following with reference to the appended drawings, in 
which: 

Figure 1 is a sectional side view of a vehicle show- 
ing the overall construction of a vehicle rain water 
drop detecting system based on the image recog- 
nition system of the present invention; 
Figure 2 is a front view of an essential part of the 
vehicle of Figure 1 ; 

Figure 3 is a flow chart for illustrating the procedure 



for detecting water drops; 
Figure 4 is a diagram showing changes in the wave- 
form of the signal as it goes through various proc- 
esses; 

Figure 5 (a) is a line data of an image captured by 
the CCD camera of the vehicle rain water drop de- 
tecting system according to the present invention; 
Figure 5 (b) is a line data of an image captured by 
the CCD camera of a conventional vehicle rain wa- 
ter drop detecting system; and 
Figure 6 is a view similar to Figure 1 showing an- 
other embodiment of the vehicle rain water drop de- 
tecting system based on the image recognition sys- 
tem of the present invention. 

DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0019] Figure 1 shows the overall construction of a 
water drop detecting system for a vehicle which is incor- 
porated with an image recognition system according to 
the present invention. A CCD camera 2 for capturing im- 
ages is provided on a reverse side of a rearview mirror 
1 provided inside a cabin of a vehicle. A screen 4 is 
placed on an upper middle part of a windshield 3 so as 
to be within the area swept by a wiper blade, to oppose 
the CCD camera 2, and not to obstruct the view of the 
vehicle occupants (Figure 2). The screen 4, for instance 
consisting of a translucent film, such as milky film, trans- 
lucent paper, frosted glass, is placed on the inner sur- 
face of the windshield 3 to disperse the light projected 
from some distance ahead of the screen 4, but relatively 
clealy shows through objects near the screen. As well 
known, such a translucent film clearly shows though any 
objects immedicately next to the film, but virtualy ob- 
struct the view of objects remote from the film. 
[0020] A hood 5 is placed over the space between the 
CCD camera 2 and the windshield 3 (screen 4) to shut 
off light from the sides, above or below, and to direct 
only the light from the screen 4 to the camera 2. The 
CCD camera 2 is connected to an image processing unit 
6 which is in turn connected to a wiper drive control sys- 
tem which is not shown in the drawings. 
[0021] Now the operation of this embodiment is de- 
scribed in the following with reference to Figures 3 and 
4. Figure 4 shows the waveform of an actually captured 
image. Normally, the CCD camera 2 sucessively cap- 
tures an image at a regular interval or in synchronism 
with the sweeping strokes of the wiper blade (step 2). It 
is assumed that a water drop R and a smear D are de- 
posited on the surface of the windshield 3, and that fea- 
tures corresponding to the water drop R and the smear 
D appear in the captured waverform. The first part of 
each cycle of the waveform is produced from the interal 
operation of the CCD camera 2, and the waveform of 
the actually captured signal is found in the range indi- 
cated by letter S. 

[0022] The captured signal is smoothed (step 2) to re- 
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move fine noises therefrom. The signal is then differen- 
tiated (step 3) to compute the change rate components 
of the signal. The obtained signal is converted into bi- 
nary data with a threshold process (step 4) to allow de- 
termination of the presence and the quantity of water 
drops and smears. The operating mode of the wind- 
shield wiper is determined according to the detected sig- 
nal, and the windshield wiper is controlled accordingly. 
However, the detected signal contains features due to 
the presence of the smear D, and, therefore, cannot be 
directly used for correctly recognizing the water drop R. 
By noting the fact that the features due to solid smears 
D give rise to larger and sustained changes of the signal 
level than the features due to water drops R, the signal 
following the smoothing process of step 2 is subjected 
to a threshold process (step 5) based on a prescribed 
level, and then to a logical product process (step 7) with 
an enlarged signal (step 6) so that the features due to 
the smear D may be removed, and the binary data con- 
tains only the feature due to the water drop R. The en- 
largement process (step 6) advances the leading edge 
of the pulse produced by the presence of a smear and 
delays the trailing edge of the pulse so that the logical 
product process can be accuately carried out even when 
some time delay is produced between the two signal 
flow lines (the signal flow line of steps 3 and 4, and the 
signal flow line of steps 5 to 7). 
[0023] Figure 5 compares the signals obtained by the 
CCD camera for the case (a) where the image is cap- 
tured by using the screen and the case (b) where the 
image is directly captured without using the screen. Fig- 
ure 5(a) clearly indicates the changes in the signal due 
to the presence of the water drops, but Figure 5(b) in- 
cludes such a large interference from the outside view 
that the determination of the presence of water drops is 
very difficult. 

[0024] The screen made of a milky translucent film 
was attached to the inner surface of the upper middle 
part of the windshield in the above described embodi- 
ment, a similar film may be placed on the outer surface 
of the windshield or between a pair of glass sheets. It is 
also possible to frost a surface of the windshield by 
physical means such as fine abrasive particles, or by 
chemical means such as a hydrogen fluoride etchant. 
Larger water drops form clearer images when the 
screen is placed on the outer surface of the windshield, 
but smaller water drops form clearer images when the 
screen is placed on the inner surface of the windshield 
because the water drops perform the function of lenses. 
The method for forming the screen is not limited by the 
placing of a film, but the screen may also be formed by 
direct vapor deposition on the windshield surface or by 
application of a heat cured coating over the surface of 
the windshield. The position of the screen may also be 
freely selected as long as it would not obstruct the view 
of the vehicle occupants, in particular that of the vehicle 
operator, as indicated by the imaginary lines in Figure 
2. For instance, when the screen is placed along the low- 



er edge of the windshield, the CCD camera 2 may be 
embedded inside the dashboard 7 so as not to obstruct 
the view of the vehicle occupants as shown in Figure 6. 
[0025] The image recognition system of the present 

5 invention was described as a rain drop detecting system 
for a vehicle such as a passenger car in the above em- 
bodiment, but the application of the present invention is 
not limited by the above described embodiment. The 
present invention can be applied to any applications 

10 where the state of liquid deposition on the surface of a 
transparent member is required to be determined. De- 
pending on the application, the screeen may be placed 
over the entire surface of the member so that the liquid 
drop detection may be carried out overthe entire surface 

15 of the transparent member. 

[0026] Thus, according to the present invention, a 
translucent screen is placed over a transparent member 
or a part thereof, the image showing through the screen 
is captured by a camera from the reverse surface of the 

20 transparent member, and the presence and the quantity 
of the liquid drops are determined from the captured im- 
age so that the image remote from the screen such as 
the external view showing through the transparent 
member are not captured, and only the liquid drops lo- 

25 cated immediately, next to the screen are extracted. 
Thus, the sensitivity and the precision in detecting liquid 
drops deposited on a member can be improved by using 
a simple arrangement. 

[0027] Although the present invention has been de- 
30 scribed in terms of a preferred embodiment thereof, it is 
obvious to a person skilled in the art that various alter- 
ations and modifications are possible without departing 
from the scope of the present invention which is set forth 
in the appended claims. 

35 

Claims 

1. An image recognition system for detecting liquid 
40 drops deppsited on a surface of a transparent mem- 
ber (3) comprising: 

a camera (2) for capturing an image; and 
means (6) for determining the presence of liq- 
45 u id drops from the image captured by the cam- 

era, 

characterised in that 

so the system further comprises a translucent 

screen (4) placed over the transparent (3) or a 
part thereof; and 

the camera (2) is for capturing an image formed 
on said screen (4) from a reverse surface of 
55 said transparent member (3). 

2. An image recognition system according to claim 1 , 
wherein said screen (4) comprises a translucent 
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sheet member placed over said transparent mem- 
ber (3). 

3. An image recognition system according to claim 2, 
wherein said translucent sheet member comprises 
a frosted sheet member. 

4. An image recognition system according to claim 2, 
wherein said translucent sheet member comprises 
a semi-transparent sheet member. 

5. An image recognition system according to claim 2, 
wherein translucent sheet member comprises a 
milky sheet member. 

6. An image recognition system according to claim 1 , 
wherein said transparent member (3) consists of a 
windshield glass sheet separating an interior of a 
vehicle from outside so that rain water drops depos- 
ited on said glass sheet may be detected. 

7. An image recognition system according to claim 6, 
wherein said camera (2) is placed on a reverse side 
of a rearview mirror (1), and said screen (4) is 
placed on an inner surface of said windshield glass 
sheet (3) opposite said camera (2). 

8. An image recognition system according to claim 6, 
wherein said screen (4) is placed on a lower edge 
of on an inner surface of said windshield glass sheet 
(3), and said camera (2) is mounted on a part of a 
dashboard opposing said screen (4). 

9. An image recognition system according to claim 2, 
wherein said translucent sheet member (4) is 
placed over an inner surface of said transparent 
member (3). 

10. An image recognition system according to claim 2, 
wherein said translucent sheet member (4) is lam- 
inated between two layers of said transparent mem- 
ber (3). 

11. An image recognition system according to claim 2, 
wherein said translucent sheet member (4) is 
placed over an outer surface of said transparent 
member (3). 

12. An image recognition system according to claim 1 , 
wherein said screen (4) is formed by frosting a sur- 
face of said transparent member (3). 



PatentansprUche 

1 . Bilderkennungssystem zum Erfassen von FIQssig- 
keitstropfen, die auf eine Oberflache eines transpa- 
renten Elements (3) abgeschieden sind, umfas- 



send: 

eine Kamera (2) zum Aufnehmen eines Bilds; 
und 

5 

Mittel (6) zur Bestimmung der Anwesenheit von 
Flussigkeitstropfen aus dem von der Kamera 
aufgenommenen Bild, 

10 dadurch gekennzeichnet, daB 

das System weiter einen durchscheinenden 
Schirm (4) umfaBt, der uber dem transparenten 
Element (3) Oder einem Teil desselben ange- 
15 ordnet ist; und 

die Kamera (2) dem Aufnehmen eines Bilds, 
das auf dem Schirm (4) gebildet wird, von einer 
Ruckseitenoberflache des transparenten Ele- 
20 mentsdient. 

2. Bilderkennungssystem nach Anspruch 1, bei dem 
der Schirm (4) ein durchscheinendes Folienele- 
ment umfaBt, das uber dem transparenten Element 

25 (3) angeordnet ist. 

3. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement weiter ein 
mattiertes Folienelement umfaBt. 

30 

4. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement ein halbtrans- 
parentes Folienelement umfaBt. 

35 5. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement ein milchig 
weiBes Folienelement umfaBt. 

6. Bilderkennungssystem nach Anspruch 1 , bei dem 
40 das transparente Element (3) aus einer Wind- 

schutzscheiben-Glasscheibe besteht, welche das 
Innere eines Fahrzeugs von der AuBenseite trennt, 
so daB Regenwassertropfen, die auf der Glasplatte 
abgeschieden sind, erfaBt werden konnen. 

45 

7. Bilderkennungssystem nach Anspruch 6, bei dem 
die Kamera (2) auf eine Ruckseite eines Ruckspie- 
gels (1 ) angeordnet ist und der Schirm (4) auf einer 
Innenflache der Windschut zscheiben-Glasscheibe 

so (3) gegenuber der Kamera (2) angeordnet ist. 

8. Bilderkennungssystem nach Anspruch 6, bei dem 
der Schirm (4) auf einem unteren Rand einer innen- 
flache der Windschutzscheiben-Glasplatte (3) an- 

55 geordnet ist und die Kamera (2) auf einem Teil eines 
Armaturenbretts angebracht ist, der dem Schirm (4) 
gegenuberliegt. 
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9. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement (4) uber einer 
Innenflache des transparenten Elements (3) ange- 
ordnet ist. 

10. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement (4) zwischen 
zwei Schichten des transparenten Elements (3) la- 
miniert ist 

11. Bilderkennungssystem nach Anspruch 2, bei dem 
das durchscheinende Folienelement (4) uber einer 
AuBenoberflache des transparenten Elements (3) 
angeordnet ist. 

12. Bilderkennungssystem nach Anspruch 1, bei dem 
der Schirm (4) durch Mattieren einer Oberflache 
des transparenten Elements (3) gebildet ist. 



R even dlcat Ions 

1 . Systeme de reconnaissance d'image pour la detec- 
tion de gouttes de iiquide d6pos6es sur la surface 
d'une piece transparente (3), comprenant 

- une camera (2) pour la capture d'une image ;et 

- un moyen (6) pour determiner la presence de 
gouttes de Iiquide k partir de I'image capture 
par la camera ; 

caracterise en ce que : 

le systeme comprend, de plus, un ecran trans- 
lucide (4) place sur la piece transparente (3) ou 
sur une partie ; et 

la camera (2) est prevue pour la capture d'une 
image formee sur I'ecran (4) h partir d'une sur- 
face oppos6e de la piece transparente (3). 

2. Systeme de reconnaissance d'image selon la re- 
vendication 1 , dans lequel I'ecran (4) comprend une 
piece en feuille translucide placee sur la piece 
transparente (3). 

3. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece en feuille trans- 
lucide comprend une piece en feuille depolie. 

4. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece en feuille trans- 
lucide comprend une piece en feuille semi-transpa- 
rente. 

5. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece en feuille trans- 
lucide comprend une piece en feuille trouble. 



6. Systeme de reconnaissance d'image selon la re- 
vendication 1 , dans lequel la piece transparente (3) 
est constituee d'une feuille de verre de pare-brise 
separant I'interieur d'un vehicule de Pexterieur de 

5 fa^on k pouvoir detecter des gouttes de pluie depo- 
sees sur la feuille de verre. 

7. Systeme de reconnaissance d'image selon la re- 
vendication 6, dans lequel la camera (2) est placee 

10 sur le verso d'un miroir de retroviseur (1), et l'6cran 
(4) est place sur une surface interieure de la feuille 
de verre de pare-brise (3) en face de la camera (2). 

8. Systeme de reconnaissance d'image selon la re- 
is vendication 6, dans lequel I'ecran (4) est place sur 

un bord inferieur d'une surface interieure de la 
feuille de verre de pare-brise (3), et la camera (2) 
est montee sur une partie d'un tableau de bord op- 
pos6e k recran (4). 

20 

9. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece de feuille trans- 
lucide (4) est placee sur une surface interne de la 
piece transparente (3). 

25 

10. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece de feuille trans- 
lucide (4) est prise en sandwich entre deux couches 
de la piece transparente (3). 

30 

11. Systeme de reconnaissance d'image selon la re- 
vendication 2, dans lequel la piece de feuille trans- 
lucide (4) est placee sur une surface externe de la 
piece transparente (3). 

35 

12. Systeme de reconnaissance d'image selon la re- 
vendication 1 , dans lequel recran (4) est forme par 
depolissage d'une surface de la piece transparente 
(3). 
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